United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 

Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-1450 
www.uspto.gov 



APPLICATION NO. 


FILING DATE 


FIRST NAMED INVENTOR 


ATTORNEY DOCKET NO. 


CONFIRMATION NO. 


09/922,760 


08/06/2001 


Takashi Mizokawa 


YAMAH6.007A 


1621 



20995 7590 09/13/2004 

KNOBBE MARTENS OLSON & BEAR LLP 
2040 MAIN STREET 
FOURTEENTH FLOOR 
IRVINE, CA 92614 



EXAMINER 



LERNER, MARTIN 



ART UNIT 



PAPER NUMBER 



2654 

DATE MAILED; 09/13/2004 



Please find below and/or attached an Office communication concerning this application or proceeding. 



PTO 90C (Rev 10/03) 



UTTice Action oummary 


Application No. 

09/922,760 


Applicant(s) 

MIZOKAWA, TAKASHI 


Examiner 

Martin Lerner 


Art Unit 

2654 





The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S. C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )□ Responsive to communication(s) filed on . 

2a)D This action is FINAL. 2b)S This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quay/e, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) IEI Claim(s) 1 to 20 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) [X] Claim(s) 1 to 20 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) Q Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) ^3 The specification is objected to by the Examiner. 

10)O The drawing(s) filed on is/are: a)D accepted or b)Q objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 

Priority under 35 U.S.C. § 119 

12)^ Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)lEI All b)D Some * c)D None of: 

1 Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1) ^ Notice of References Cited (PTO-892) 

2) O Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) ^ Information Disclosure Statement(s) (PTO-1 449 or PTO/SB/08) 

Paper No(s)/Mail Date 8/6/01 & 2/12/02 . 



4) □ Interview Summary (PTO-413) 

Paper No(s)/Mail Date. . 

5) □ Notice of Informal Patent Application (PTO-1 52) 

6) □ Other: . 



U.S. Patent and Trademark Office 

PTOL-326 (Rev. 1-04) 



Office Action Summary 



Part of Paper No./Mail Date 092004 



Application/Control Number: 09/922,760 Page 2 

Art Unit: 2654 

DETAILED ACTION 



Specification 

1 . The disclosure is objected to because of the following informalities: 

On page 4, lines 4 to 8, the copending United States Patent Applications should 

be updated, as follows: 

Serial No. 09/393,146 is now U.S. Patent No. 6,604,091 issued 05 August 2003. 
Serial No. 09/736,514 is now U.S. Patent No. 6,629,242 issued 30 September 

2003. 

Serial No. 09/129,853 is now U.S. Patent No. 6,230,111 issued 08 May 2001. 
Serial No. 09/630,577 is now U.S. Patent No. 6,347,261 issued 12 February 

2002. 

On page 5, line 15, Figure 5 is a block diagram, not a flow chart. 

On page 9, line 24, "base" should be -based — . 

On page 13, line 9, "be" should be deleted before "suggested". 

On page 1 8, line 5, "4f ' should be -2f— . See Figure 2. 

On page 21, line 23, "neumatic" should be -pneumatic — . 

On page 22, line 6, "neumatic" should be -pneumatic — . 

On page 26, line 13, "S1 10" should be -S100— . See Figure 6. 

On page 27, line 7, "4j" should be -4i — . See Figure 2. 

On page 28, line 1 , "5c" should be -5b — . See Figure 2. 

Appropriate correction is required. 
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Claim Rejections - 35 USC § 102 

2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

3. Claims 1 to 3 f 5, 6, 9 to 11, 13 to 15, and 20 are rejected under 35 U.S.C. 102(b) 
as being anticipated by Henton. 

Regarding independent claims 1, 5, 9, 13, 14, and 15, Henton discloses a 
method and apparatus for automatic generation of vocal emotions, comprising: 

"a sound data memory which stores a different sound assigned to each pseudo 
emotion" - Table 2 gives examples of the defined emotions with their associated vocal 
emotion values; the values in Table 2 are relative to the default neutral speech setting 
(column 9, line 56 to column 10, line 40: Table 2); computer system 10 includes 
memory 17 (column 5, lines 18 to 35: Figure 1); implicitly, values of emotions shown by 
Table 2 are stored in memory 17 of computer system 10; 

"a sound signal generator which receives signals from the pseudo-emotion 
generator and accordingly generates a sound for each pseudo emotion by retrieving the 
sound data stored in the sound data memory" - after a particular emotion has been 
chosen 503 ("receives signals from the pseudo-emotion generator"), the appropriate 
speech synthesizer values are obtained via look-up table 505 ("generates a sound by 
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retrieving sound data stored in the sound data memory") (column 9, lines 47 to 55: 
Figure 5: Steps 503 and 505; column 4, lines 38 to 59); 

"a sound synthesizer which is programmed to synthesize a sound by combining 
each sound signal from the sound signal generator, wherein the user can recognize 
overall emotions generated in the interaction device" - appropriate speech synthesizer 
values are applied by embedding the appropriate speech synthesizer commands to be 
output by the text-to-speech system (column 9, lines 47 to 55: Figure 5: Step 507; 
column 4, lines 38 to 59); 

"an output device which outputs a synthesized sound to the user" - computer 
system 10 has output 21 , which implicitly includes a speaker for producing audio to 
output synthetic speech by a text-to-speech system. 

Regarding claims 2, 6, and 10, Henton discloses a plurality of emotions ("multiple 
sets of sound data") are stored by Table 2, with each emotion represented by 
parameters for pitch, volume, and speaking rate (column 10, lines 17 to 40: Table 2); a 
particular vocal emotion is chosen 503 ("a selection device which selects a set of sound 
data to be used based on a designated selection signal") (column 9, lines 41 to 55: 
Figure 5: Step 503). 

Regarding claims 3 and 1 1 , Henton discloses emotions are defined by a 
speaking rate (column 10, Iines17 to 40: Table 2); emotions for a concatenative speech 
synthesizer involve parameters including speech rate and duration (column 3, line 54 to 
column 4, line 1 1: Table 4; column 12, lines 1 to 14: Glossary); thus, broadly, both 
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duration and speaking rate involve a "designated selection signal indicating the passage 
of time." 

Regarding claim 20, Henton discloses emotions are defined by a volume varying 
between 0.2 for low to 0.8 for high from a default of 0.5 (column 10, lines 17 to 40: 
Table 2; column 4, lines 1 to 1 1 : Table 1 ; column 1 1 , lines 1 to 42); thus, Angry2, 
Aggressive, and Happy are given greater intensity than Sad or Tired ("produce an 
acoustic effect equivalent to the intensity of the pseudo-emotion"). 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claims 4, 8, and 12 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Henton in view of Saito. 

Henton generally discloses an interactive device, insofar as computer system 
10 permits a user to interact with a speech synthesizer to select emotions, but omits a 
designated selection signal indicating the history of interaction between the user and the 
interactive device. However, it is known for games and toys to draw on a history of 
interaction with a user to determine a personality for a game or toy. Saito teaches a 
method for controlling character based electronic game development, where initial 
values of personality elements of a player character are determined before the main 
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part starts, but when a pre-event is executed, initial values of the personality elements 
which correspond to the type of pre-event data can be increased or reduced. Thus, 
when a player character gets a ring for a girl, initial values of personality elements 
relating to a sense of beauty are increased, and when a player character plays baseball, 
initial values relating to exercise ability are increased. (Column 9, Line 35 to Column 
10, Line 6) The objective is to improve video games in which scenario-based contents 
are developed by permitting a game player more naturally to recognize scenario-based 
developments and to smoothly change characteristics of a player character. (Column 2, 
Lines 1 to 20) It would have been obvious to one having ordinary skill in the art to 
select emotions in an interactive device of Henton by a signal indicating a history of 
interaction as taught by Saito for the purpose of permitting characteristics to naturally 
develop in a scenario-based video game. 

6. Claims 7 and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Henton in view of Edatsune ('488). 

Concerning claim 7, Henton discloses a method and apparatus for generation of 
emotions by user interaction with a computer system 10, but omits a growth stage 
selection unit to select an artificial growth stage based on the passage of time by a 
signal from a growth stage calculating unit. However, Edatsune ('488) teaches a toy 
providing interactive speech recognition with varying responses, where a response level 
value is gradually increased to provide an illusion that a stuffed toy is growing up like a 
living creature as time passes. Thus, a stuffed toy can only respond with "Bow-wow" to 
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"Good morning" on the first day after being purchased, but on the second day, it can 
respond "G-o-o-d mor-ning" to "Good morning", and after several days, the stuffed toy 
can respond with "It's a nice day, isn't it?" to "Good morning" because of a higher level. 
(Column 12, Lines 12 to 56: Working Example 3) The objective is to enable a more 
sophisticated interaction by a toy in response to speech issued by a user. (Column 1 , 
Lines 30 to 46) It would have been obvious to one having ordinary skill in the art to 
provide a growth stage selection unit for calculating a growth stage based on the 
passage of time as taught by Edatsune ('488) in a method and apparatus for generating 
emotions of Henton for the purpose of enabling a more sophisticated interaction by a 
toy in response to speech issued by user. 

Concerning claim 16, Henton discloses a speech synthesis method and 
apparatus for generating emotions, where any stimulus to produce emotions is provided 
by a user through keyboard or touch screen input 13, but omits a stimulus recognition 
device for recognizing stimuli from the outside. However, it is well known, generally, for 
interactive voice response (IVR) systems to provide speech recognition and speech 
synthesis, and, specifically, for a toy to include interactive voice response (IVR) so as to 
recognize speech and respond with synthetic speech. Edatsune ('488) teaches a toy 
providing interactive speech recognition with varying responses, where microphone 1, 
speech analysis unit 2, and speech recognition unit 5 recognize input speech, and 
speech synthesis unit 6 and speaker 8 output the content of speech synthesized to the 
outside. (Column 4, Line 40 to Column 5, Line 5: Figure 1 A) Also, a response content 
creation unit outputs responses by taking into consideration environmental temperature 
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and air pressure. (Column 3, Lines 1 to 10) The objective is to enable a more 
sophisticated interaction by a toy in response to speech issued by a user. (Column 1 , 
Lines 30 to 46) It would have been obvious to one having ordinary skill in the art to 
provide a stimulus recognition device for taking into consideration speech, temperature, 
and air pressure and for outputting responses as taught by Edatsune ('488) in a method 
and apparatus for generating emotions of Henton for the purpose of enabling a more 
sophisticated interaction by a toy in response to speech issued by user. 

7. Claims 17 to 19 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Henton in view of Kibre et a/. 

Henton discloses a method and apparatus for generation of emotions, where a 
table provides for correspondence between registered emotions and sound (column 10, 
lines 1 7 to 40: Table 2), but omits a character forming device for forming any of a 
plurality of characters, a growth stage specifying device for specifying growth stage, and 
a table selection device for selecting any of a plurality of sound data correspondence 
tables. However, Kibre et a/, teaches a computerized personality system with a text-to- 
speech engine, where words and phrases may be tagged to indicate a linguistic context 
of which voicing parameters to use. (Column 2, Line 58 to Column 3, Line 18: Figure 1) 
Specifically, parameters used by the system provide for speaker types of human, alien, 
robot, and animal ("a character forming device for forming any of a plurality of 
characters"), and an age parameter for baby, toddler, child, teenager, middle-aged 
adult, and old adult ("a growth stage specifying device for specifying growth stage"). 
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(Column 8, Lines 1 to 19) Each linguistic context specifies a set of parameters defining 
a character type or growth stage, analogous to values for emotions in Table 2 of 
Henton. Thus, each linguistic context is represented by a table of values ("sound data 
correspondence tables"). The objective is to provide a sophisticated text-to-speech 
engine that automatically generates natural-sounding spoken messages that enhance 
the simulation of a human attendant by selecting different types of expressions under 
different circumstances. (Column 1, Lines 30 to Column 2, Line 2) It would have been 
obvious to one having ordinary skill in the art to provide a character forming device, a 
growth stage specifying device, and a table selection device as suggested by Kibre et 
al. in a method and apparatus for generating emotions of Henton for the purpose of 
automatically generating natural-sounding spoken messages that enhance the 
simulation of a human attendant. 

Conclusion 

8. The prior art made of record and not relied upon is considered pertinent to 
Applicant's disclosure. 

Breese et al. discloses related art. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Martin Lerner whose telephone number is (703) 308- 
9064. The examiner can normally be reached on 8:30 AM to 6:00 PM Monday to 
Thursday. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Richemond Dorvil can be reached on (703) 305-9645. The fax phone 
number for the organization where this application or proceeding is assigned is 703- 
872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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